Model Selection, Estimation, and Bootstrap Smoothing

Classical statistical theory ignores model selection in assessing estimation accuracy. Here we consider bootstrap methods for computing standard errors and confidence intervals that take model selection into account. The methodology involves bootstrap smoothing, also known as bagging, to tame the erratic discontinuities of selection-based estimators. A projection theorem then provides standard errors for the smoothed estimators. Two examples, non-parametric and parametric, are carried through in detail: a regression model where the choice of degree (linear, quadratic, cubic,. . . ) is determined by the Cp criterion, and a Lasso-based estimation problem.